Reproducible research in archaeology: Basic principles and common tools

Ben Marwick, University of Washington
Society of American Archaeology Meeting, April 2015

Definitions are in flux

alt text

Computational - Statistical - Empirical
Stodden, V., et al. 2013. “Setting the default to reproducible.” computational science research. SIAM News 46: 4-6.

Motivations: Claerbout's principle

“An article about a computational result is advertising, not scholarship. The actual scholarship is the full software environment, code and data, that produced the result.” - Claerbout and Karrenbach, Proceedings of the 62nd Annual International Meeting of the Society of Exploration Geophysics. 1992

Claerbout's principle in detail

“When we publish articles containing figures which were generated by computer, we also publish the complete software environment which generates the figures” - Buckheit & Donoho, Wavelab and Reproducible Research, 1995.

“The scholarship does not only consist of theorems and proofs but also (and perhaps even more important) of data, computer code and a runtime environment which provides readers with the possibility to reproduce all tables and figures in an article.” - Hothorn et al., 2009 Case studies in reproducibility

Goal is to expose the reader to more of the research workflow

alt text http://www.stodden.net/AMP2011/slides/pengslides.pdf

Current workflow in archaeology & many other disciplines

  • Enter data in spreadsheet/database
  • Use Excel for data cleaning & descriptive statistics
  • Import data into SPSS/SAS/Stata for further analysis
  • Use point-and-click options to run statistical analyses
  • Copy & paste output to Word document, repeatedly

alt text

My workflow & tools for reproducibility

Using literate statistical programming

Using an open source programming language & open document formatting language

Using version control & continuous integration

Using dynamic documents

Using an isolated environment

Using permissive licenses & permanent URLs

alt text

alt textalt text

A case study: An executable article

We must integrate these values into everyday tasks

  • Train students by putting homework, assignments & dissertations on the reproducible research spectrum
  • Publish examples of reproducible research in our field
  • Request code & data when reviewing
  • Submit to & review for journals that support reproducible research
  • Critically review & audit data management plans in grant proposals
  • Consider reproducibility wherever possible in hiring, promotion & reference letters.

“Abandoning the habit of secrecy in favor of process transparency and peer review was the crucial step by which alchemy became chemistry.”

-Raymond, E. S., 2004, The art of UNIX programming: Addison-Wesley.

Colophon

Presentation written in Markdown (R Presentation)

Compiled into HTML5 using RStudio

Source code hosting: https://github.com/benmarwick/

ORCID: http://orcid.org/0000-0001-7879-4531

Licensing: